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Abstract. The problem of estimating tlie shift (or, equivalently, the center 
of symmetry) of an unknown symmetric and periodic function / observed 
in Gaussian white noise is considered. Using the blockwise Stein method, a 
penaUzed profile likelihood with a data-driven penalization is introduced so 
that the estimator of the center of symmetry is defined as the maximizer of 
the penalized profile likelihood. This estimator has the advantage of being 
independent of the functional class to which the signal / si assumed to belong 
and, furthermore, is shown to be semiparametrically adaptive and efficient. 

Moreover, the second-order term of the risk expansion of the proposed 
estimator is proved to behave at least as well as the second-order term of the 
risk of the best possible estimator using monotone smoothing filter. Under 
mild assumptions, this estimator is shown to be second-order minimax sharp 
adaptive over the whole scale of Sobolev balls with smoothness /3 > 1. Thus, 
these results extend those of |10| . where second-order asymptotic minimaxity 
is proved for an estimator depending on the functional class containing / and 
/3 > 2 is required. 



1. Introduction 

1.1. Motivation. The increasing interest to the semiparametric models in recent 
years is mainly explained by the compromise they offer between the relative sim- 
plicity of parametric inference and the flexibility of nonparametric modeling. In 
many semiparametric models, though the ignorance of an infinite dimensional nui- 
sance parameter, the finite-dimensional parameter of interest can be estimated as 
well as if the nuisance parameter were known. In most situations, there are many 
estimators having this feature. Moreover, most of them depend on the functional 
class to which the unknown nuisance parameter is assumed to belong. 

The aim of the present paper is to propose a second-order efficient and entirely data- 
dependent estimator in the problem of shift estimation when the observed shifted 
signal is corrupted by the Gaussian white noise. This problem has been studied 
in [TU], and the goal of the present paper is to complete the inference developed 
there. In particular, the estimator proposed in [TU] and proved to be second-order 
minimax when the signal belongs to a Sobolev ball, depends on the parameters of 
the Sobolev ball in question. In the present work, we propose an estimator of the 
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shift parameter that is independent of the parameters of the Sobolev baU and is 
second-order minimax simultaneously for a large scale of Sobolev balls. 

To further motivate the study of the second-order efficiency in semiparametric in- 
ference, let us briefly recall a popular general method of construction of efficient 
estimators in a semiparametric model: the profile likelihood maximization. The 
asymptotic properties of this method are studied in [32j and, in a more general 
fashion, in f24^ Profile likelihood techniques are effectively applied in a number 
of contexts such as laser vibrometry signals [22] , varying coefficient partially linear 
models [13], and so forth. 

If the parameter of interest is partitioned as {■&; /), with i? being a low-dimensional 
parameter of interest and / a higher dimensional nuisance parameter, and /) 
is the log-likelihood of the model, then the profile likelihood for ■& is defined as 
plni'd) = siipf^-p Ini'd; f) and the Profile Likelihood Estimator (PLE) is dpLE = 
argmax^ pZn,(i?). Thus, the nuisance parameter / is eliminated by taking the sup 
over all possible values of / in some a priori chosen class T. 

A natural question arises: what is the best way of choosing the class F and what is 
the impact of this choice on the accuracy of the PLE? The theory fails to answer 
this question as long as only the first-order term of the risk is considered. It seems 
that the most appealing way to study the dependence on T of the accuracy of the 
PLE is to consider the second-order term of the quadratic risk. This approach is 
developed in [161 IE] for partial linear models, in [TOj for a nonlinear model with 
a shift parameter, and in [7] for a model with scaling parameter. In these papers, 
the second-order asymptotic minimaxity of the proposed estimators is proved and 
optimal constants are obtained. 

Another important point is that the study of the second-order asymptotics of the 
risk allows one to refine the choice of the tuning parameters, if there are, of the 
estimating procedure. This argument is used in [19[ I21[ 123] in order to propose a 
rate-optimal choice of a tuning parameter. 

The results of the above mentioned papers grant an increasing importance to the 
second-order terms in that they show that, in a semiparametric estimation prob- 
lem, the second-order term is not dramatically smaller than the first-order term, 
especially when the nuisance parameter is not very smooth (or not very sparse). 
Thus, the investigation of second-order efficient estimators is not only a challenging 
theoretical problem, but is also of practical interest. 

1.2. Methodology. The main goal of the present paper is to define an estimator 
of d which adapts automatically to the smoothness of / and is simultaneously 
second-order efficient over a large variety of Sobolev balls. For linear models, such 
a procedure has firstly been proposed by Golubev and Hardle in [17]. They use 
the well known idea of unbiased risk minimization in order to determine the data 
driven filter. However, their procedure is provably second-order efficient only when 
the (data driven) filter and the contrast function are based on independent sub- 
samples. This sample-splitting technique is frequently used in theory (see e.g. [2], 
[26[ p. 19]), but it is rather unattractive from the practical point of view. 

In this paper, we focus on the shift estimation of a periodic symmetric signal and use 
the celebrated Stein shrinkage (see [20l[29]) for defining the data-driven version of 
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the penalized profile likelihood. While there is a huge number of papers concerning 
the Stein's shrinkage in nonparametric estimation ( [U El [6l O O [HI [28j , see also 
[34j for a comprehensive discussion) , this is to our knowledge the first paper where 
Stein's shrinkage is applied in the context of semiparametric estimation. 

The construction we use is closer to the one of jB^ in that a penalized version of 
the Stein estimator with weakly geometrically increasing blocks is considered. It 
seems that, unlike in nonparametric inference, in semiparametric inference it is 
crucial to use the penalized shrinkage in order to get the second-order efficiency. 
An important advantage of using the Stein Shrinkage is that, unlike the procedure 
proposed in [171, our procedure is not based on the sample-splitting technique and, 
nevertheless, enjoys the property of second-order asymptotic minimaxity. 

Note also that our results hold when the function / has arbitrary Sobolev smooth- 
ness (3 > 1. However, for /3 < 2, the penalized profile likelihood (PPL) is not 
necessarily concave over the whole parameter space, therefore we use a two-step 
procedure. We first construct an initial y^-consistant estimator -d of "i?, and then 
maximize the PPL over a shrinking neighborhood of where we prove that the 
PPL is concave with probability close to one. 

1.3. Structure of the paper. Section 2 describes the model and introduces the 
Penalized Maximum Likelihood Estimator (PMLE) based on a filtering sequence 
h. In Section 3, the local concavity of the PPL is proved and the PMLE based on 
a data dependent choice of h is proposed. Oracle inequalities for adaptive PMLE 
and its second-order efRciency over Sobolev balls are stated and proved in Section 
4. Finally, Section 5 contains the definition of a preliminary estimator and the 
technical details of the proofs. 

2. A SIMPLE SEMIPARAMETRIC MODEL 

Consider the "signal in Gaussian white noise model", that is the observations 
(x=(i), t e [-1/2,1/2]) with 

dx%t) = U{t)dt + edW{t), [-1/2,1/2], (1) 

are available, where W{t) is a Brownian motion. Assume that the signal has the 
form f^{t) — f{t — I?), where / : R — + M is a symmetric periodic function having 1 
as smallest period. More precisely, we assume that the function / belongs to the 
set J^o = Up>o^o(p) with 

Mp)^{feLl, : f{x) = f{-x) = f{x + l), VxeM; |/i|>p}, 

where we denote by Lf^^ is the set of all locally squared integrable functions and 
by /i - I-{%fi-t) cos{2TTt)dt. 

The goal is to estimate the parameter i? e c] — T, T] with T < 1/4. As explained 
in [TU] , the assumption T < 1/4 is necessary for the identifiability of the parameter 
•d. In this context, the unknown function / is considered as an infinite dimensional 
nuisance parameter. 

The first-order asymptotic properties of estimators in closely related models have 
been studied in [TSl [HI [3T1 [HD]- Note also that the model we have just defined is 
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an idealized version of the symmetric location model [33l [23] and the shifted curves 
model [T4] . 

Let us introduce the "sequence model" equivalent to H]). To do this, for any integer 
k> 1, we denote 

.1/2 

Xk = \/2 / cos{2nkt)dx^{t), 

^-Ijl (2) 

-1/2 



V2 [ sm{2TTkt)dx''{t). 

J -1/2 



Clearly, 



Xk = fk cos(27rfci9) + e^fc, , . 

xl = fkSmi27rk^)+eCk, ^' 



where fk — j]/^^.^cos{2'iTkt) J{t) dt and {^k,Ck,k — 1,2,...) are independent 
standard Gaussian random variables. Note that, the laws of likelihood processes 
(indexed by {d, /) G R x J^q) of the models ([J) and ^ coincide implying thus the 
equivalence of these models. It is useful to mention here that the Fisher information 
in the problem of estimating 1} with fixed / is 



= f'ixf dx = e-' ^(27rfc)V|. 

•^^1/2 ken 



In this paper, we estimate the parameter i9 by a version of the method of profile 
likelihood maximization (cf. [351 p. 106]), which is also known as penalized maxi- 
mum likelihood estimator [TUj. We recall briefly its definition. Write P^j (resp. 

j) for probability measure (resp. expectation) induced by x^ on the canonical 
space C([— 1/2, 1/2]) equipped with the Wiener measure. As no confusion is pos- 
sible, we use the same notation in the "sequence model" given by ([3]). The Radon 
density L^{t, /, •) of P^j with respect to Po,o is given by 

1/2 ^-2 /.1/2 



L,(t, /, x') = cxp ( e-' I fit - t) dx%t) - — / fit) dt 

-1/2 ^ J-1/2 



= c'^P E ^ / cos[27rfc(t - r)] dx^t) -^J^f' 
fci ^-1/2 ^£ k=i 

Easy algebra yields 

max Le(T,/,a;^) = expje^^^ /" cos[2TTk{t ~ t)] dx" [t)) \. (4) 

For any r € M, this expression is equal to infinity for almost all paths x". Thus, 
it is necessary to impose some restrictions on the set over which the maximization 
is done. This is equivalent to considering a penalized profile likelihood. In this 
paper, we focus on the penalization corresponding to ellipsoids in More details 
on this method can be found in [lOj , we here content ourselves with giving the final 
definition. 
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We call filtering sequence or filter any h = {hk)keN G [Oj I]'' such that only a finite 
number of hk are non-zero. Define the functional 

$^(r,/i) = y^hki cos[2nk{t- T)]dx''{t)] . (5) 

fc=l Vi-i/2 / 

The PMLE of is then ??pmle = arg maxT- $e (r, /i) . The role of the sequence h is 
thus to filter out the irrelevant terms in the right side of (jl]), that is to assign a 
value hk close to zero to the terms corresponding to a small signal-to-noise ratio 
\fk\/s. 

For deterministic filters /i, the asymptotic behavior of the estimator ??pmle is studied 
in [10]. Under some smoothness assumptions on /, for a broad choice of filters h, 
i^PMLE is proved to be first-order asymptotic efficient. Moreover, it is shown that 
the second-order term of its risk expansion is e'^R'^[f,h]/\\f'\\'^, where 

oo 

R^f,h] = ^(2^fc)2[(l-M'A'+£'/^^]. 
fc=i 

This result suggests to use the filter hopt = arg min/i [f, h] for defining the PMLE 
of "d. However, this minimizer is inapplicable since it depends on f. To get rid 
of this dependence, the minimax approach recommends the utilization of the filter 
hyr = arg min/i supjg^ R'^lf, h]. If ^ is a ball in a Sobolev space, a solution of this 
minimization problem is given by the Pinsker filter [27]. Although the latter leads 
to a second-order minimax estimator of i9 [TOl Thm. 2 and 3], it suffers from the 
well known drawbacks of the minimax theory: the obtained estimator is pessimistic 
and requires the precise knowledge of the smoothness of the unknown function. 

The aim of the present paper is to propose a data-driven filter h so that the re- 
sulting PMLE of circumvents these drawbacks. To be more precise, let us call 
oracle related to the class of filters TC the value argmin^^.^^ ft,]. We cannot 
use this oracle in our estimating procedure because it depends on the unknown 
/. Nevertheless, the estimator we propose mimics well the behavior of the oracle 
related to the classe of monotone filters (cf. Section HT^)) for a fixed function / and 
is second-order sharp adaptive over a broad scale of Sobolev balls. 



3. PMLE BASED ON A DATA-DRIVEN FILTER 

3.1. Local properties of ^^(t, ft). Let us introduce some auxiliary notation: 

J-l/2 
J -1/2 

We will write Xk = a;fe(0), x*^ = x^(0) and similarly for and The symmetry 
of / implies that Xk{'&) = fk + £^ki^) and xK'd) = e^l{-d). Moreover, for all t e M, 
the random variables {^^(t), ^* (r); i G N} are i.i.d. standard Gaussian. Using this 
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notation, the functional $£ can be rewritten as follows 
1 °° / 

$^(t, /i) = - ^ /i J xki-ff) cos[27rfc(T - d)] + xli^) sin[27rfc(T - i?)] 



2 

k=l 



Our aim is to show that under some assumptions on h, the function h) has, 
with a probability close to one, a local maximum in a neighborhood of Note that 
the derivative of the function t i-^ $j (r, h) is given by 



$^(t, /i) = - ^ /ifc7rfc(xfc(i?)2 - xli^f) sin[4^fc(T - t?)] 
fc=i 

oo 

+ ^ 2hkTikxk{'d)xl{'d) cos[47rfc(r - (6) 

/c=l 

Proposition 1. Let ll/^''*'!!^ = Y.A'^'^kf'^' fl < °° V 

some /3* > 1 and set 

/3* = /3* A 1.5, iVe = [(e^ loge~^)~2]riTT]. Let /i e [0,1]^' be a random vector 
depending on {xk,x'^) only via x'l + a;^^. For any e < 1, there exists an event 
such thatP^jiA"^) < 2e^ and on n { ^^^(1 - /ifc)(27rfc)2/2 < ||/'||V4}, for all 
s verifying 

' - 2||/('3.)|| +67r' ^'^ 

the function t i— s- <i>e(T, h) is strictly concave and admits a unique maximum i?^ in 
the interval [d - {4:TrN^y^,-d + (AnN^)^'^], satisfying 

<8£ v/biF^ II /'ir^ (8) 

Proof Set = [i? - i4:nN,)~\i} + {4:7rN,)~^]. Assume that © is fulfilled and 
- hk){2'Kkffl < ||/'||V4. On the one hand, the first inequality of Propo- 
sition [4] (see Section [521 below) implies that there exists an event Ae, such that on 
this event 

Kir, h) < -WfT + - hk){27rkrf', + II/'IIV4 < -||/'f /2, 

fc=i 

for all r e 9e and for e small enough. Therefore, is strictly concave. On 

the other hand, the second inequality of Proposition |4] implies that 



Kir,h) ^ _\\fT 2£(||r||v/b^+2^jV^^») 
r-i9 - 2 \T-d\ 



JlfT^W^EllIl fa, 
- 2 ^ \T-d\ 

for sufficiently small values of e. Therefore, ±$^(i? ± (47riVg)~^, /i) < 0, which 
guarantees that the maximum of is attained in the interior of O^ and 

i>^(4, ft.) = 0. Applying © to t = 4 we get ©. □ 

Remark 1. The choice = [(e^ loge"^)^ ^^j+t] has a simple interpretation. If 
fk ^ £^ foi' some k G N, then the fcth observation in ([3]) is not relevant for estimating 
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the parameter Let /C = {fc e N : /| > e^} and K = #/C. Then 

keK k£K k=l ^ * 

Thus, the number K of Fourier coefRcients fk larger than e is at most 0(e^ 213, +i )^ 
Thus, for £ small enough, all observations relevant for estimating the parameter d 
lie in {yi, . . . ,yAr}. 

Remark 2. In [ID], the estimator i^pmle is defined as the maximizer of 
over the whole interval Q. Instead, we define it as the local maximizer in the 
neighborhood of a preliminary estimator. This modification is explained by the fact 
that the function r 1-^ ^e{T^ h) is only locally concave when < 2. Furthermore, 
the computation of the local minimum is faster than the computation of the global 
minimum. 

Remark 3. Elaborating on the arguments of Proposition [l] it can be shown that 
Thm. 1 from [lOj remains true for /3 > 1, provided that 'iJpMLE is defined as the 
local maximizer of /i) and condition B2 is replaced hy hk = for any k > N^. 

3.2. Blockwise constant Stein filter with penalization. Let J be a positive 
integer and ki, . . . , kj+i G {1, . . . , Ng,} be a strictly increasing sequence such that 
Ki = 1. Set Bj = {k e N : Kj < k < Kj+i}. Let H*{B) be the set of ah filters 
h E [0, 1]^- that arc constant on the blocks B = {Bj}j^^: 

hen*{B) hk = hk', yk,k'eBj. 

An oracle knowing the function /, would choose the best possible filter h* G TL*{B) 
by minimizing R^[f^h] over Ti*{B). We call h* oracle choice of filter or simply 
oracle. Simple computations show that 

II f IP 

"fc— ||f,||2 I £.2^2' Kt-Dj, J — 

11/ II (j) 

Where \\f\\l) = T.keB.i'^^kf fl and = Y.keB.i'^^kf. 

Since the oracle h* depends on /, we replace it by a suitable estimator. Let 
us define R1[f,a] = (1 - af\\f'\\l^ + e^a^a^^ so that R'[f,h] = + 
^j^yjq {2nk)'^f^. Then h^. is the minimizer of Rj[f,a] over a e M. Define 
Wy'Wu) = EfceB,(27rfc)^|2/fcp and yk = Xk + ixl. For smah values of /i*^., the 
minimizer of 

R^[a] := (1 - af{\\y'\\l) - 2eV|)+ + e^a^al 

which is an estimator of [/, a] , can be large with respect to /i* ^. . To avoid such 

a configuration, we penalize large values of a and define the estimator h'-'^ of h* as 
the minimum over [0, 1] of the function: 

where (pj > is a factor of penalization tending to zero as e 0. This leads us to 
the penalized Stein filter 
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3.3. Weakly geometrically increasing blocks. The ami of this section is to 
propose a concrete scheme for defining the blockwise constant data-driven filter. We 
use the weakly geometrically increasing blocks introduced by Cavalier and Tsybakov 
[H [9] . These blocks have the advantage of being simple (the construction is driven 
by only one parameter) and of having good approximation properties with respect 
to the class of monotone filters f9', Lemma 1] . 

— 1/3 

Let 1/ ^ he a positive integer that increases as £ decreases. Set pe = Ve and 
define 

\K,_l + KPe(l + Pep-'J, J -3,4,..., 

where [zj stands for the largest integer strictly smaller than x. Let J be the 
smallest integer j such that nj > + 1. We redefine = A^^ + 1 and set 

Bj = {kj , . . . , Kj+i — 1} for all j = 1, . . . , J. 



3.4. Brief description of the procedure. The outlined scheme can be imple- 
mented as follows. 

1. Choose a real number /3* > 1 and set A^^ = 5 V [(e^ loge~^)~ ^ffirn"]^ i/^ = 

2. Define the sequence {Kj)j by pT|) . 

3. Set = \/241og£-V(Kj+i - Kj), cr| = EK,<fc<K,+i (27rfc)2 and define the 

data-dependent filter ft'^'^ by ([T0|) . 

4. Compute the preliminary estimator (cf. Section [5|) and set 8e = [ds — 

Se, + Se] with 6, = e l0g(£~2j. 

5. Define ??e as the minimum in 8^ of h"'^) (see (O). 

Note that the only "free" parameter in this procedure is In practice, if no 
information on the regularity of / is available, it appears plausible to assume that 
/ has Sobolev smoothness /3* = 2. 



4. Main results 



4.1. Comparison with the blockwise constant oracle. In this section, h'^^ 
denotes the blockwise constant filter defined by pO]) . ipj is the penalization we 
use on the block Bj and = max^ ipj . We emphasize that in this section no 
condition on the blocks Bj is required. Let Tj be the length of the block Bj and 
= infj Tj. The oracle choice of h in the class Ti*{B) of all filters constant on the 
blocks B — {Bj}j is denoted by h* . Define 

^^•s = argmax$e(T,/i'^'^), (12) 

where = [dg — 5^,d^ + 5^] and is a rate optimal initial estimator of (cf . 
Section [5TT|) . Introduce the functional class 

^(/3*,i*,p) = j/ e Mp) ■■ ll/'^-'ll < i*}, 
where /?* > 1, p > 0, > are some constants. 
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Theorem 1. Let i}^^ be defined by U2]} with blocks Bj verifying logs ^ — o{Te) 
as 



e ^ 0. If the penalty (pj is equal to y24Tj ^loge ^, then 



e-W^,j[0f - m < 1 + (1 + ^^)^jpp^ 
where ^ as e — > uniformly m / G ^(/3*, L^, p). 

Remark 4. If the block Bj is large, then more observations {xk,xl) are used 
for estimating the value of the oracle /i* . . Hence, it is natural to expect that 
decreases as increases. A thorough inspection of the proof allows to describe 
this feature with the help of the order relation x loge~^. 

Proof Let us denote £ = {hi^ G [hi - ^/^,^pJ(k){^ - hi), hi], Vfc = 1, . . . , N^} and 
h = h'^^l-s, where j{k) is the number of the block containing k. Lemma [5] implies 
that P^j(£'^) < 2£^. For e small enough, we have (1 — hk)t£ < 2(1 — hi) and the 
inequality ^^^(1 - hk){2Trk)^f^ < /4 is fulfilled on £. 

By virtue of Proposition [31 for sufficiently small values of e, the event Aq = ||^?£ — 
'd\ < ( j^ - Se) A {6e - ^'^ l uq ' )} verifies Pi),f{A^) = 0{e^). It can be checked 



that on Aq, 



i9± 



C [-de ± ^e] C ± 47]v~]' where we have used the 



ll/'ll 

notation [i^ ± S] -.^ - S,d + S]. 

According to Proposition [H on the event Ai — AoHAenS, the function ^^{■,h) is 
strictly concave and has a unique maximum in [de ± Se]. Hence the estimator i)^^ 
verifies h) = on the event Ai- By Taylor's formula, there exists a point 

^ G I'd, S^'^] such that 

= h) = <i>^(^, h) + idi' - d)<P':{d, h) + h). 

Therefore, on Ai , 

^Jii _ ^ = 



h) + - h) 

Using ([6]), one checks that 



$;'(^, h) = -J2 hk{27Tk)'[{fk + eU^))' - e'CkW'' 

k=l 



In the sequel, we write £,k,£,k instead of ^fe('!9), (i?). On the one hand. Lemmas 
[SI [7] (with x'^ — log£~^) and [5| combined with (|^^ imply that, on an event A2 of 
probability higher than 1 — 4e^, we have 



$^'(^, h) = -WfT + - K)i^^k)^fk - 2eY,Ki^^k)^Mk + o{Rn 

k=l k=l 

= - wrr'R'if, h*]-c- o{R'[f, h*])). 



10 



A. S. DALALYAN 



where ( = 2£||/'|| '^J2k^*ki'^^^)'^fk(,k is a zero mean Gaussian random variable. 
By virtue of (fT6|) . its variance verifies 

4.^£/.f (2.fc)V| < ^tK^mi^ < '"'^'"y^^'"*^ - (13) 
fe=i ^ ^ j=i ^ 

Therefore, by Rosenthal's inequality, E^j[(^p]^/p = o{R'^[f,h*]) for any p > 0. On 
the other hand, in view of Lemma [HI there is an event A3 such that Ptf,/(.4§) = 
0(e4) and 

on this event. Using the inequality — < l + 2x + 16x^ for all x G [—1/2, 1/2], 
we get 

mm' ~ = , 

{i-\\r\\-'mf,h*]-c-o{R^)Y 

< hf (1 + + 2C + + o{s'R^), 

on the event A4 — Ai D A2 H A3. Using Lemma [51 we infer that 

00 

fc=l 

= e'{\\.rr-R'[f,h*]). 

Combining these relations with Lemmas [10] and lll[ we get an event A such that 
F^jiA'') = 0{e^) and 

||/'||4e,,^[(4^^ - ^)H^] < hf] (^1 + o{e'R^[f, h*]) 

<emT+e^RnLh*]+o{e^R-[f,h*]). 

Since - i?| < 1, we have E^jH-ffi^ - '&)^1a''] < P^jiA") = 0{e'^). In view 
of (1231), ^o{R'[f,h*]). Therefore Ei, J 19)21^0] = 0(e4) = o(e2i?=[/, /i*]) 
and the assertion of the theorem follows. □ 

4.2. Comparison with the monotone oracle. Now we consider the class Timon 
of filters having decreasing components, that is 



'^mon 



{h e [0, 1]^- : hk > hk+i, l<k<N,-iy 



The class Timon is of high interest in statistics because it contains the most common 
filters such as the projection filter, the Pinsker filter, the Tikhonov or smoothing 
spline filter and so forth. 

Proposition 2. Set 'y^ — maxi<j<j_i(CT|_|_]^/cr|). Then 

R'if, h*] < 7e ,^^inf R'if, h] + eV?. 

A more general version of this result is Lemma 1 in Since the proof in our 
setting is simple, we give it below. 
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Proof. Let h he a filter from Timon- Define h hy hk — h^. if fc e Bj for some j and 
/ifc = if fc > iVg. Since the components of h are decreasing, we have l — h^ < ^~hk 
and therefore 

J 

R^[fM<Y.^l-huf{2'Kkfll+e^Y.^%^l (14) 

Again by monotonicity of h, we have /i^j < hk for all A: G Bj^i. Hence, 
J J 

j=2 j=2 k=l 

Combining this inequality with (fT4|) and bounding hi by 1, we get R^[f,h] < 
7eii!^[/, /i] + af. Since h e H* and /i* minimizes -R^[/, /i] over aU h G H* , we have 
/i*] < 7ei?^[/, /i] + e^CT^. This inequality holds for every h £ Hmon, therefore 
the assertion of the proposition follows. □ 

Combining this proposition with Theorem [1] we get the following result. 
Corollary 1. Assume that the conditions of Theorem [1] are fulfilled, then 

e-W^,j[iSi' - ^f] < 1 + 7e(l + ^e) ""^^^-f^^''l 

where ^ as e uniformly in / e J-{f3^^ , L^,, p). 

Remark 5. For the blocks defined by (fTTI) . we have = Vgp^{l + p^), a\ < 
A'n'^vl and —i^ePe + ^e{^ + PeV < < 1 + Ve{^ + PeY ■ One also checks that 

7j = maxj crj_|_i/cr| is asymptotically equivalent to (1 + pc^Y 1 + 3pe as e ^ 0. 
Therefore the factor in the oracle inequality of Corollary [1] is of order (1 + 3pe + ae). 
We have already mentioned that a1 = 0{T~-^ loge~^). The trade-off between 

— 1/3 

and pe leads us io pe^ Ve ■ This clarifies our choice of pe slightly differing from 
the one of [9]. 

Remark 6. In [8j[28l[34] the weakly geometrically increasing blocks are defined by 
= \y{^ + Py~^\ ■ This type of blocks docs not lead to a sharp oracle inequality 
in our case, since we need not only iaa,y:.{Tj+i/Tj) — > 1, but also max(Kj+i//tj) — > 1 
as £ ^ 0. 

4.3. Second-order minimax sharp adaptation. To complete the theoretical 
analysis, we show below that the estimator d'l^ corresponding to the blocks pT|l 
enjoys minimax properties over a large scale of Sobolev balls. Assume that / e 
T{(3\L*,p) and define 

2 

Theorem 2. Assume that verifies e^i^+^v^ — s- as e — > and the conditions of 
Theorem [7] are fulfilled. If 5 = tends to zero as e — > and f € J-{P* ,L*,p) with 
P* > P > P*, then the estimator -d^^ defined in Section \3.4\ satisfies 



4/j-4 

sup _ e-2||/'||2E,,,pf -^)^] < l + (l + o(l))-^(^'^)'''" 



in- 
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2/j-2 



whene^O, withC{(3,L) = 1(2^^^^)^^ {L{2f3+l))^i^ . Moreover, the follow- 
ing lower bound holds: 

inf sup E,,^[(4 - ^f] > 1 + (1 + 

where the inf is taken over all possible estimators i?e . 

Proof. According to Lemma [T2| there exists a filter A* e Timon such that 
sup i?^[/,A*]/||/'||2 < (f + o(f))C(/3,L)£^sup||/'|r2. 

Since (5^ ^ as e — > 0, we have snpffzyr^ ^ ^(/) ||/'||~^ = (l+o(l))||/'||~^- Combining 
this result with Corollary [U we get the first inequality. The second inequality is 
Theorem 2 of [10]. Although the latter is stated for (3 > 2, the inspection of its 
proof shows that the same claim is true for any /3 > 1. □ 



5. Preliminary estimator and technical lemmas 

5.1. Preliminary estimator. Having the observation {x^{t), \t\ < 1/2), we can 
compute xi,xl by Then we have xi — fi cos(27ri9) +e^i and xl — fi sin(27ri?) + 
eQ, where ^i, are independent standard Gaussian random variables. We define 

i/^ — — arctan — , 
2tt \xij 

if Xl 7^ and = 1/4 if xi = 0. One easily checks that is the maximum 
likelihood estimator in the model induced by observations (a;i,a;^). The following 
result describes its asymptotic behavior. 

Proposition 3. If e is sufficiently small, then 

sup P-i)j{\de-^\>x) <exp(-2(a;/e)Vf cos2(27rT)), 

for all X G [0, 1/2] and for all T < 1/4. 



Proof. Let us introduce X = \J + H"^ ■ One checks that sin[27r('(^e — -d)] = 
sin(27ri9£) — cos(27n?£))/-|"^. The Cauchy-Schwarz inequality implies that 
|sin[27r(t9e - ^)]\ < e^l/f^l- Since G [-1/4,1/4] and e [-T,T], we have 



^ c°«(2'^^)- Therefore, 



T, /,n n, ^ I |sin[27r(i9e -i3)]| 



27rcos(27rT) 
< Pi).f{X > 22;£-V|/i| cos(27rT)), 

and the fact that Ar^/2 follows the exponential law completes the proof. □ 
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5.2. Proofs of Lemmas used in Proposition [H Let us start with some basic 
facts that will be often used in the proofs. For any n,m,p e N, we have 



n^{n — to) > > 



k—m+l 

Applying this inequality to p = 2, we get 

max(27rfc)2 < 3cr?/T,, 

fcG-B, ■' 



nP(n — to) 
p+1 



(15) 



(16) 



Assume now that ^ is a random variable of law Af{0, 1). For any < 1/4, we have 
(1 - 20-2)-! < 2 and (1 - 2a^)-^/^ < e^''\ therefore 



p(l-2cr^) M I o_2 



< exp (2/1^ + 20-2 



(17) 



Vl - 2(72 

Using the more precise inequalities (1 — 20-2)^^ < 1 + 4(t2 and log(l — 2ct2)^^ < 
2a-2 + 4ct'', we get E[e(''+'^*)^] < exp(^2 +0-2 + 2o-2(2^2 ^ ^2^^ equivalently, 



Throughout this section, we assume that H/'-'^'-'lP — X^fcC^'^'^)^''* /| < for some 
/3* > 1, A^e = [(e2 loge~^)~2fj.+i] and /i € [0,1]^^ is a random vector depending 
on (xfc, x^) only via + x*f^^ . Without loss of generality, we give the proofs in the 
case d = Q. 

Lemma 1. Set (3^ = (3^ A 1.5. For all r such that 47r|T - ■&{ < N~'^ , 



<^2-2/3./||^(,.)||^27rA/v/biF^) , 



fe=i 

where X = maxi<fe<Ar^ 

Proof. One easily checks that 

$"(i? + T, /i) = - ^(27rfc)2/ifc/2 cos[47rfcr] 
fc=i 

A, 



2e^(27rfc)2/ife/fc(efe cos[47rfcT] + Ck sin[47rfcT]) 



A, 



£2 ^(2^fc)2/ifc [(^2 - Cf) cos[47rfcr] + 2^:.^ sin[47rfcr]] . 



fc=i 



On the one hand, thanks to inequality |1 — cosx| < |a;|, 

^(27rfc)2/ife/2(l - cos[47rfcr]) < 2r^(27rfc)3/; 



<2r(2^Ar,)3-2^*||/(^*)|l2 

< A^2-2/3.||j(/3.)||2^ 
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On the other hand, in view of the Cauchy-Schwarz inequahty, it holds ^fe cos[47rA:T] + 
CfeSin[47rfcr] < X and {(i - C^^) cos[47rfcT] + 2^^^^ sin[47rfcT] < . Therefore, it 
holds 

^(27rfc)2ftfc/fc(^feC0s[4^fcT] +efcSin[47rfcr]) < X ^(2^fc)2|/fe| 

k=l k=l 



<1||/(W|| 



. ^(27rfc)4-2/3. < 2ttXN^ '^*||/'''*^||, 

\ k=l 



and 



^(2^fc)2/ifc [ifk - if ) cos[4^fcT] + sin[47rfcr]] 



fc=i 



< 47r2iV,3^^ 



Taking into account the identity e'^N^ — ^^*/ log(e for all r verifying |t| < 
{iTTNi^y^, we get 



fe=i 



< ^2-2^.(||^(/3.)||2 ^ 2^X/v/foiF^) = 



and the assertion of the lemma follows. 



□ 



Lemma 2. Let h e [0,1] ' be a random vector depending on (a;fe,a;^) only via 
+ xf . For any x G [0, it holds 

Pi)j(\^',{d,h)\ > 2xe{\\f\\+27:eNf')) < 26"^' 



Proof. The random variables X^ — (27rfc)(/fe+e^/c)^^, k = 1, . . . , and hi, . . . , h]\f 
fulfill the conditions of Lemma [Til with = l,Tk = l/(2V27rfce), gl = (27rfc)2(/| + 
e^), since due to (fT7|) . 



By definition, ^'^{d, h) — eJ2k=i ^kXk, and therefore. 



(19) 



[W,{^M>2xe{j2{2^kf{fl+e^) 



k=l 



1/2 



< 2e-^ 



for all X e [0, (Sfc(27rfc)2(/2+e2))i/2/(2V27rA^e£)]. To complete the proof, it suffices 
to remark that 



Ek=ii2^knPk+e'} > Ek=ik' ^ 



1,2 



2N? 



and EtiC^^kfifk+e') < (ll/'ll +£(27r)7Vr^) 



3/2x2 



□ 



Proposition 4. Assume that ||/'''^*|| < oo for some /3* > 1 and set /3* = /3* A 1.5. 
There exists an event Ae such that for every e < 1/2, P^j{A£) > 1 — 22^ and on 
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Ae it holds: 



/..(2.fc) VI + iVr^'^* (11/(^*^11 +37r)^ 

k=l 



, 2g(||,n|Vb^+2^iV^/^») 

/or an r e [i? - (47rAre)" \ + (47rA^e)~^]- 
Proof. According to Lemma [U we have 

K{r,h) < - ^/.,(2^fcf /I + A^r'^* (11/^^*^11 +27rl/Vl^)'. 

Since for every fc, (C^ + ^fe^)/2 follows the exponential law with mean 1, we have 

P(4X2 > 91oge~5) < N,'P{Xl/2 > loge"^) < N,e^ < e*. 
This inequality completes the proof of (PO)) . 

To prove ((2T|) . note that for some f € [^,t], we have $^(t, /i) = /i) + (t — 

Lemma Hand (EHl) yield (EH). □ 

5.3. Lemmas used in Theorem [H Let us start with some simple algebra allow- 
ing to obtain a rough evaluation of h*], where h* is the ideal filter an oracle 
would choose in the class of blockwise constant filters. For this filter h* , it holds 



,/ 2^211 «||2 

^.^1 11/ ll(j) CTj fc>jv. 

Using the explicit form of h^, we get 

RV, h*] = ^(1 - K){2^kf,fl >s^Y. Ki^Trkf- (22) 



AT, 



fc=l fc=l 



Since e'^N^ as £ ^ 0, we have R''[f,h*] < CiV|-2/5. ^ q as e ^ 0. In view 
of ||/'||(i) > 4:71^ and e^af < An^e^N^ ^ as e — > 0, for e smah enough the 
inequality e^cTi < holds. Therefore, 



-2^2 



Hence, for every function /, the quantity /i*] tends to zero as £ ^ slower 

than £^ and faster than e^N^. 

Lemma 3. It holds ^[^'^{d,hf] < e^'^hf{2TTkf{f^+e^). 

k=l 
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Proof. Using ([6]), one checks that 



(24) 



In the sequel, we write ^k,£,k instead of ^k{^) , Cti^) ■ For any k' ^ k, the random 
variable hk{fk + £Cfc)Cfc''-fc' (/fc' is symmetric. Therefore it has zero mean 

and 

Em^rh)'] = e'Y.{2nkrE[hl{fk+e^k?Ck'] 
<e'J2(^7Tky^f-E[ifk+eikra']. 



k=l 

and the assertion of the lemma follows. 
Lemma 4. Let us denote 



Vj 



^^EkeB, {2^WMk + EkeB, i^nkfiek + - 2) 



For any positive x such that < Tj/lO, it holds 



Pi\rj,\>xJ24{l-h%jT-'] <2e-- 



□ 



Proof. Set Yfc = 2e(27rfc)2/fe^fe + £^(277^)2(^2 _^ _ 2), (t = e{2TTk)Vt and /i = 
(27r/^)/fe^/^. Using (HH]) we infer that 



as soon as cr = e{2iTk)\/t < 1/2, or cquivalently t < 1/Ae'^{2nk)'^. By [25l Thm. 
2.7], for any 2: > 0, we get 



where 



It is clear that 



4(2£7rK 



(8E.eB,(2'rfc)^)i/^ ^ (8(2^)4^4 ^j../5)i/2 



n > — ^ > 

- 4(27rKj+i)2 - 4(27rKj+i)2 

^ (2.fcr(A2 +£2) < {2nn,+,n\\f%^+s'a^) < 3Tr^a^i^f%^ + s^af, 

keBj 



< 



m-hl^){\\f\\l^+e^a]r 



r,£2 



These inequalities combined with the identity rjj = J2keB- +^ "j) yi^ld 

the desired result. □ 
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Lemma 5. Assume that (pjTj > 24 log e ^ and (pj < 1. The 



P( hfe 



■ 2(1 ~ hl)^, ■ 



(1 - ^j) 



> 1 - 2£^ 



Proof. Note that 



(25) 



One checks that /i^'^ > /i^ if and only if rjj > ipj. Therefore, P(/i^ > 
P(77j > (fij) < e"^. Similarly, 



(1 - Vj) 



since /i^ > ^'j^^ if and only if ipj — (1 + (pj)hl < —ifij- Therefore, using 

Lemma |4l 



= P{ipj- {1 + tpj)hl < T]j < -ipj 



+ Pivj < - (1 + Vj)K)'^,,,^ 



<P(77, <-^,), 



P( hi^ e 



(1 - v'.O 

and the assertion of the lemma follows 



1 "fc 



□ 



Lemma 6. For any positive x verifying < it holds 



>l^^R^f^h*] ] <2Je-"' 



Proof Using (HH), we get (Efees, (27rfc)'*)^^^ < ia'j/y/T). This inequahty com- 
bined with the fact that hk < h^., allows us to bovmd the probability of the event 
of interest by 

J 



k=l 



>V32x^/.:^(^(2vrfc)^) 

3=1 keBj 

<Ep(^«. ^(2^fcf(e^-ef) >y32x/»:^.(^(27rfc)'^)'^'] 

j=l ^ k<£Bj keBj ' 



j=l ^ k^Bj keBj 

The desired result follows now from Lemma [T3l and ([T5 
Lemma 7. For any x > 0, 



□ 



Pe 



Y,{hk-hl){2nkf^kfk 



k=l 



>5xJ^ R'[f,h*]\ <2Je- 
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Proof. Remark first that 

- hif < A 8(1 - Kf^'^ < V8^,/i:^.(i - K^) 

for all j = 1, . . . , J. Set Yj = e J^keBj {'^^k)^^kfk- The random variables Yi, . . . ,Yj 
are independent zero mean Gaussian with variance 



Therefore, P(|5^- 1 > -^6/7} xecTj II /'II (j)) < 2e '-^ and consequently, 

^\Y.\^K - K,)Y,\ >^-xeJ2^Aip,h%^il ~ /i*^KI|/'||(,)j < 2Je--r 
To complete the proof, note that 

and the right side is bounded by R'^[f, h*]. □ 

Lemma 8. We have ^(/ife - hi) {2t: kf f I < 4:ip,R''[f,h*]. 
fe=i 

Proof. The desired inequality is trivially fulfilled on S'^, while on £ we have < 
K-hk < 4:ip,{l-hl), \fk e Bj, and hence ^ \hk - hl\{27rkf f^ < iip^Y^il- 
hl){2^kffl<A^,W[f,h*]- □ 



Lemma 9. Set X ~ maxi<j.<7v^ VCfeW+^fe W- There exists an event of prob- 
ability at least 1 — 2e^ such that on this event, for all t G M,. we have 



|a>f(r,fe)| ^ 12|r-z?|.||r|p 12||/1|v/fo^ 
R^[f,h*] - e^T, eTe 



sVTe 

Proof. Using ©, one checks that 

oo 

<f^"(T, h) = 2Y,hk{2nkf[{fk + e^km' ~ e'Cki^f] sin[4^fc(T - ^)] 

oo 

- 4£ ^ hk{2TTkf{fk + eUmm cos[4^fc(r - i?)]. 
fc=i 
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Without loss of generality, we assume in the sequel that = 0. Then 
|$;"(r, h)\ <2Y,hk{2nkffl\ sin(47rfcr)| + 'ieY,hk{2Trkf\fkCk\ 



k=l 



k=l 



+ 4e^/ifc(27rfc)3|/fe[^fesin(47rfcT) +Cfc(cos(47rfcr) - 1)] | 

k=l 

+ 2e2 hf,{27rkf\iek - ) sin(47rfcr) - 2^^^^ cos(4^fcT) | 



fc=i 

Using the inequalities | sin(47rfcr)| < 47rfc|T| and 



fc=l J = l " 3 = 1 



as well as the inequahty X]/=i ^kj^I — we get the desired bound for 

the first sum. The bound on the second term is obtained using Lemma I14[ the 
well known bound on the Laplace transform of a Gaussian distribution and the 
inequality 



k=l 



< 



J^2 

= 1 " j=l 

" maxfe* < "-^ " , ' — ^. 



The bounds on the two remaining sums are obtained by combining the inequalities 
l^fc sin(47rfer) + ^fc(cos(47rfcr) - 1)| < |47rfcT| • \£_k cos(47r/cf) - sin(47r/cf)| 

< 4:nk\T\X, 
I (42 _ ^f) smiAnkr) ~ 2^kCk cos(4^/ct)| < 
with arguments similar to those used to bound the first two sums. 
Lemma 10. For any x > 0, it holds 



□ 



h) - h*) > 12e^ x'^if.R'if, h*] ] < 2e-=^'(i^V^^78^). 



Proof. Let us denote Xk — {2nk){fk + e^k)£.k- According to 



$:.(^, h) - $^(1?, h*) = sY^ih - hl)Xk. 

According to for aU t < I / {2y/2Tike) , we have E[e*-^^] < e^^-^^^f Ul+^'') _ Thus 
the conditions of Lemma [Til are fulfilled with — hk — h^., g^. — 3a//i^(/Jj(1 — h^), 
Tk = l/(\/8 7rfc£) and gl = (27rfc)2(/| +e'^). Therefore, 

2 \ 
$^(^, /i) - $;(^, r ) > 12e2a;Ve E(27rfc)2/i* (1 + j < 2e--' 

k=l ' 
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for any a; > verifying 



^2 ^ iT.k=ivmi'^^kfKi^~K)Ul + ^^) 

8e2 maxj- ipjhl^ (1 - /i*^ )(27rKj+i)2 
To complete the proof, it suffices to remark that 

Y,{2irkfhl{l - hl){fl+e^) = - K^)[\\r\\l-^+e^a^] < R^[f,h*] 

k=i j=i 

and 



> 



Se^maxj - /i*^.)(27rKj+i)2 8 maxj ^j/i*^. (1 - /i*^.)(27rKj+i)2 ' 

Since aj > Tj{2TTKj+if' /i, the assertion of the lemma follows. □ 

Lemma 11. Let ( = 2e\\f'\\^'^^f,h*f,(2Trk)'^fk£,k- For any event A verifying 
PiA") = 0{e^), we have 



Proof. We have 



r 



= 2e^E 



=1 

AT, -, 1 

J2i27:kfhff,U = 4£4||/'||-2^(2^A:)4/if /2 
fc=i -I 
J 

< 12r,-V5]/j:/a2 < 12T,-i£2^-[/,r]. 



Using the Rosenthal inequality, one easily checks that E[<i>^(i?, /i*)2p] = 0(£2p) 
and E[C^^'] = o{R'^[f, h*]P) for any integer p > 0. Therefore, the Cauchy-Schwarz 
inequality yields. 



|E[$;(^, h*)^aA^] I < o{e'^R-[f,h*]) = o{s^VR'[f, h*]) 

and the assertion of the lemma follows. □ 

5.4. Lemma used in Theorem [2l We assume that / e ^s,p,L{f) with / e 
jF(/3* ,L*,p) and /?* >/?>/?*. For the sake of completeness we give below a 
suitable version of the Pinsker theorem [27j . 

Lemma 12. Set -f, = l/loge-2, W, = ^(fffig^i} ) '^^'^^'^ <^rid define 



1, 



A: \/3-i' 



, k > 7eWe. 



T/ie filter At satisfies 



sup i?^[/,A*] < (1 + 0(1)) C(/3,L)£WT, 
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Proof. Set V = f - f. Using the inequality (/& + Vk)^ < 2z"Vfc + (1 + z)vl, 
Vz G [0, 1], we obtain 

oo 

<2z-i E (27rfc)2(l-A^)2/2 + (i + z)i?-[^;,A*]. 

Since / G T{/3*,L*,p), we have 

E (27rfc)2(l ~ ADV"^ < L*{j,W,r-'^' = o{Wt''). (26) 

On the other hand, setting A^, = (1 — (k/W^)^^ 

R'[v, A*] - R'[v, A*] < e' E (^Trfc)' < 47r2e2(^^p^^)3 ^ o(e2pF3). 

Using the relation H/?-2/3 _ O(e^W^) and choosing z = appropriately, we get 

sup i?^[/,A*] < (1 + o(l)) sup R''[v,X*]+o{e^W^), 
/e^6,3,i.(/) i>eW(/3,L) 

where W(/3,L) is the Sobolev baU {v : Efe>i(27rfc)^^/| < i}. It then follows 
from [U Thm. 1 and Example 1, p. 265] (with a = /3 — 1 and ^ = 3) that 
sup„gyy(^^) A*] = C(/3, L) e'^TT^i + o(l)). To conclude, it sufhces to remark 

4/3 — 4 

that o(£2M/3) ^ pj-j.aTJTT). □ 



5.5. Auxiliary general results. 

Lemma 13. Assume that ai,a2, . . . , a„ e K bkc? ^ — '^k{£,k ^ Cfe^); where 

(Cii ■ ■ • I Cn, Ci ; • ■ ■ J o zero mean Gaussian vector with identity covariance ma- 

trix. For any y G [0, |la||/ max^ Uk], it holds 

p(,^>32y2Ea?,) <2e-^\ 

^ k=l ' 



Proof. Using the formula of the Laplace transform of a chi-squared distribution, 
for any t e [-(%/8afc)"\ (v^Ofc)"^], we get 



1 - 4a2t2 

Applying ^25, Thm. 2.7] with gk = 16a^ and a; = -s/Mj/HaH , we get the desired 
result. □ 



Lemma 14. Let Ai,...,A„ be independent symmetric random variables. Let 
Q — (Qi 1 ■ ■ ■ J Qn) be o. random vector satisfying \gj\ < g* , Vj = l,...,n with 
some deterministic sequence (pp"=i and ^{Q\Xj — x) — ^{Q\Xj — —x) for 
allj e {!,..., n}. // 
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for some sequence {gj)j=i,...,n and for \t\ < Tj, then 

2 



(n 2 n \ 

.7 = 1 .7 = 1 ^ 



3 

where Q„ = (Ej=i fi'j^ffj)^^^ ^^(^i/Pj)- 



Proof. Set 1^ = gjXj and Ij = g'jXj. For any pi, . . . ,p„ G N", the expectation 
E[y/'^ • . . . • FP"] vanishes if at least one pj is odd. Therefore, mJ2j Yj)''] = if A: 
is odd and HiJlj Yj)''] < E[(^^. if is even. Hence 



E 



(2fc)! 

E[e*^.-e;'^^] < e*'^^ V|t| < min(rj/e*). 



J 

According to the Markov inequality, for every t > 0, 

P( E^iX, >y)<2e-'yE[e'^^^q<2e^p(-ty + t'J2sf9^)- 

V j=i ^ ^ i=i ^ 

Setting t = (x A 0„)/(Ej ^^^^^ and j/^ = 4x^J2j sfdj we get the desired 
result. □ 

Aknowledgement. We are thankful to the anonymous referee for the remarks 
that helped to improve the presentation. 
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